ENGSTRAND & HALL: THE USE OF STREAMED LECTURE RECORDINGS 


The use of streamed lecture recordings: 
patterns of use, student experience and 
effects on learning outcomes 

Susan M Engstrand, Susanne Hall 
Moray College 

University of the Highlands and Islands 
Sue.Engstrand@uhi.ac.uk 


Abstract 

We describe a study conducted during 2008-2009 that investigated three issues relating to the use 
of streamed video recordings of classes by students at the University of the Highlands and Islands. We 
investigated (i) the pattern of use of the resource, (ii) the student experience and (iii) the impact of 
watching the streamed classes on learning outcomes measured by retention of factual content. We found 
that most (61 %) of the recordings were being used by students, with a peak in usage mid-semester. 
Students were mainly using recordings to replace class attendance on occasions when they could not be 
present, though routine use of the resource by fully online students was also common. Significant technical 
issues reduced the quality of the experience, although the experience was still reported as positive by most 
(65%) of students who had used the resource. All students who had not used the recordings reported that 
they would be interested in such a resource. We could find no difference in retention of factual content 
between our sample of students who attended classes compared with those who viewed a recording, 
however our sample was small. We suggest that this technology should be fully adopted, and recommend 
elements of development that should be addressed to maximise the full benefits to learning. 
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Introduction 

Many institutions are now investing heavily in the technology that enables the recording of lecture 
material for distribution in a media file which can be disseminated either over the internet ('streamed') or 
downloaded (podcast) for playback by students. Science, engineering and computer-based courses have 
adopted the technologies most readily (Hew, 2009). These technologies provide clear opportunities for staff 
and students, but to enhance the learning experience, there needs to be an understanding of the way in 
which it is used. 

The University of the Highlands and Islands (UHI) links over 8,000 students dispersed across the highlands 
and islands of Scotland. Teaching and learning are structured around a blend of face-to-face elements 
and distance provision. Some courses are delivered fully face-to-face, while others are delivered fully 
online. Many courses employ a mixture of delivery modes within the programme of study. This mix 
permits flexible learning in terms of time, place and pace for many students, but it also presents complex 
challenges, relating to the differences in pedagogy of distance versus face-to-face education. Key features 
of effective learning (achieving active engagement, group participation and cohesion, frequent interaction 
and feedback and presenting ideas in real world contexts) are achieved using different structures and 
pedagogies in face-to-face versus online learning (McFarlane, 2011). 

In this study, we explore the pedagogical issues relating to one aspect of technology (the streaming of 
video conference classes) which forms part of the toolkit of resources available to lecturers at UHI to foster 
learning in a diverse student cohort. 

To date, there has been little evaluation of streamed lecture recordings, with podcasts, mainly audio 
rather than video, receiving more attention than streamed recordings (Salmon and Edirisingha, 2008, Hew 
2009). As the distinction between streamed media and podcasts relates to the mode of distribution of 
material rather than inherent differences in content, results may have some relevance across the breadth of 
technologies. 
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In general, studies have found that students are enthusiastic to use recorded materials and that their 
voluntary uptake of such resources is high (Copely, 2007; Salmon and Edirisingha, 2008). However, staff 
often voice concerns about the limited possibility for interaction with such resources (Woo et al., 2008). 
Students generally report a positive experience of using these web-based learning technologies (Copely, 
2007; Pilarski et al., 2008; Woo et al., 2008) although technical issues with sound or audio quality were 
issues for some streamed recordings (Akiyama et al., 2008). 

Evidence suggests that students mainly use podcasts to aid revision for examinations (Copely, 2007; White, 
2009; Hew, 2009). A positive impact on assessment performance has been associated with viewing online 
recordings (Akiyama et al 2008, Wieling and Hofman 2010). Despite the concerns of lecturers (Woo et 
al., 2008), no evidence has been found to show that class attendance falls with the availability and use 
of lecture recordings (Copely, 2007; Hew, 2009; Nast et al., 2009; White, 2009). Students report the 
advantages of regular scheduled study and the opportunity to ask questions as being important factors in 
influencing their decision to attend classes, if they can (Copely, 2007). 

Video recordings of lecture material have been less widely used than audio, in part because of the 
additional complexities and hardware requirements needed to record video at most institutions (Copely, 
2007). However, for institutions that routinely use video conference (VC) technology these barriers are 
much less of an issue. The UHI links students from 13 academic partners and many additional learning 
centres with their tutors and peers via a combination of technologies, with video conferencing being 
a major component of the blend of learning in many programmes. In 2007, the university adopted 
technology (Codian IPVCR 2200) to initiate a pilot project to permit recording of VC classes and 
subsequent distribution to students via a web link. Recordings are made at the request of the tutor, capture 
the elements of the VC class (tutor, students, visual material, e.g. PowerPoint slides) and are stored for 
two weeks. However, during our study, server space permitted recordings to be stored for the course of 
the semester. Tutors provided the web link and a unique code for each recording via the virtual learning 
environment (Blackboard). 

The purpose of this research was therefore to contribute to building an understanding of the use of 
recorded lecture material by students, particularly within the specific context of video conferencing 
employed at the UHI. To do this, we identified three aims: 

1. to investigate the pattern of use of the recordings. 

2. to describe the student experience of using the recordings. 

3. to evaluate the impact of using recordings on retention of factual information. 

Methodology 

Pattern of use 

The number of completed playbacks of recorded video classes and the date on which the playback took 
place was collated and analysed to establish the pattern of use during January-May 2009 (semester 2). 
Downloads were excluded from the analysis as students were thought to be unlikely to download such 
large files. Thus we may underestimate the extent of the use (157 downloads recorded). 

A total of 578 of the 2201 video conference classes (26%) held during this semester were recorded, 
spanning 47 different courses. Recording length reflected the most frequent VC teaching slot (1 hr 27 min 
35 sec, s. d. 0.08 minutes). 

Survey of student experience 

An online survey (Survey Monkey) consisting of a mix of 24 open and closed questions (responses recorded 
on a Likert scale), was emailed to all current enrolled students at the UHI (approximately 8000 students) 
from November 2009 to January 2010. 

A USB stick was offered to a randomly chosen participant as an incentive to participation. Data were 
treated confidentially and anonymously. 
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The impact of watching the streamed classes on learning outcomes, assessed by retention of 
factual content 

We investigated how effective watching the recorded classes was as a learning tool by assessing 
the retention of factual content by students as a variable which we considered to be relevant to the 
achievement of learning outcomes. However, clearly this does not capture all elements of learning. 

The study involved voluntary participation from students taking a first-year ecology module at UHL 
The cohort of 34 students were geographically dispersed across the highlands and islands, with some 
attending weekly VC classes, while others were distance learners, studying fully online using the virtual 
learning environment (Blackboard). The study was explained to the whole class (34 students) before they 
volunteered, and a book voucher was presented to one random participant as an incentive. All data were 
anonymised. 

Nineteen students who chose to participate in the trial were randomly divided into two groups and the trial 
conducted over two classes in April 2009. 

• Group A students attended the first class (termed 'attend') by participating in the video 
conference session. They did not attend the second class, but instead viewed the streamed 
recording of the class (referred to as 'view'). 

• Group B were asked to 'view' the streamed recording of the first class, then 'attend' the VC for 
the second class. 

All students were then asked to take a short (17 questions) factually based quiz delivered through the 
virtual learning environment (VLE) (Blackboard) 7 ± 2.7 (n = 14) days after either attending the class or 
viewing the recording. This 'matched pairs' design aimed to limit the influence of individual variation in 
performance and thus to maximise the power of the test with a small sample (Sokal and Rohlf, 1994). 

Results 

Pattern of use 

Sixty one percent (354 recordings) of recordings had at least one playback. The median number of 
playbacks per recording was 2 (IQR 0, 7), with a maximum of 53 playbacks for a single recording. As the 
number of playbacks may be expected to vary with class size, the number of completed playbacks for each 
recording was expressed per student enrolled on each module, where these data were available. The mean 
number of playbacks per student was 0.78 +/- 1.5 per head, n = 564 (median = 0.1 57). 

To examine the temporal pattern of use throughout the semester, a mean weekly value for the mean 
number of playbacks per student was calculated across modules. There was a trend towards increasing use 
during the early part of the semester, peaking mid-semester, represented by the polynomial regression line 
in Figure 1. 

Figure 1. 



Mean number of completed playbacks per enrolled student during the course of the semester. Excluding a two week 'reading period' (weeks 11-12, 
when only one module held classes), the data are described by a quadratic polynomial function: 

(F2,16 = 4.075, p = 0.037, y = 0.158x - 0.007x2 +0.100, adjusted r2 = 0.225). 
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Survey of student experience 

A total of 175 students responded to the survey. This low return rate (approximately 2%) was a 
consequence of our inability to target students with experience of delivery by VC. However, this issue was 
addressed within the survey by filtering the appropriate student groups for questioning according to their 
responses. 

Sample profile 

Our sample comprised mainly undergraduate degree students, with some participation by HNC/D and 
postgraduate students. 55% of respondents said that either their current or a previous course had included 
some teaching by video conferencing. Many (58%) of these students were aware that VC classes were 
recorded in their programme of study, although 33% said they didn't know whether this facility was used 
or not. 

Accessing the recordings 

Of those students who knew that recordings were being made for their programme, 65% knew how to 
access them, the remainder either having some knowledge but not feeling confident (19%) or having 
no knowledge (16%). Most of these students had watched the recordings, either once or twice (40%), 
or regularly (at least once per week, 31 %). The majority (94%) accessed the recordings on their home 
computer, with only a small minority using college equipment (6%). Most (77%) of students found they 
had the appropriate software to access the recording, although recordings played smoothly for only 46%. 

Reasons for using the recordings 

Students not aware of the recordings were asked whether they would be interested in such a resource 
and all (100%) indicated that they would. Students were asked to select the main reasons why they would 
be interested in using the recordings and to rate these on a three-point scale. Students who already used 
recordings were asked to rate their reasons on the same scale. For both groups, the main reason was 'I 
can catch up on occasional classes that I cannot attend' (Table 1). Subsidiary reasons included the ability to 
navigate within the recordings, and the opportunity for recordings to be used as a revision aid. Students 
indicating that 'other' reasons for non-use were important cited: 

• routine difficulties in attending classes due to work commitments; 

• difficulties in hearing or taking notes in class; and 

• specific support needs which made it difficult to participate in VC classes. 


Answer Options 

Response Count 

for students 

using the 

resource 

Response Count 

for students not 

currently using 

the resource 

1 can catch up on occasional classes 

that 1 cannot attend 

84 

88 

1 can study at a time which suits me 

67 

72 

1 can use the recordings as a 

revision aid prior to exams 

75 

85 

The recordings are useful for my 

specific learning requirements which 

make it difficult to take notes in class 

46 

41 

1 study fully online and rarely / never 

attend a college or learning centre 

37 

47 

These recordings give me a chance 

to see tutor or classmates 

38 

46 

1 like having the flexibility to move 

backwards and forwards (to recap 

on important points) 

74 

85 

Other 

24 

13 


Table 1. 

The response rating count (calculated as number of responses by importance, very important = 3, important = 2, least important = 1) indicating the 
main reasons for using the streamed VC recordings, for 35 students currently using the resource and for 35 students not aware of the resource. 
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Playing the recordings 

Most students who used the recordings found the experience 'positive' (51 %) or 'very positive' (14%), 
although the experience was 'negative' for 17% and 'very negative' for a further 9%. Technical issues 
(buffering, recordings not playing fully or poor sound or picture quality) were by far the main issue reported 
as detracting from the experience. 


Teaching and learning 

Tutors at UHI are encouraged to adopt a seminar or tutorial approach to structuring VC sessions, rather 
than a more traditional lecture which is not thought to engage students across the VC. Students were 
asked to identify the format of material included in the recorded classes and also to identify formats 
that they would like to see in classes. The results (Table 2) indicated that most classes contain a lecturer 
delivering content to the camera, but illustrations such as PowerPoint slides and other images are also 
commonly used, as are group discussions and question-and-answer sessions. Students would like to see 
more multimedia, e.g. video, and fewer discussions or question-and-answer sessions in their recorded VCs. 
While some students did not feel part of the class and missed opportunities for interaction, others felt the 
recordings gave a good opportunity to see and hear discussions from other members of the class. 


Answer Options 

Classes 

contain 

Would like 

to see... 

A talking head (lecturer delivering content directly to the 
camera) 

85.7% 

62.5% 

Lectures illustrated with documents, PowerPoint slides, 

images 

74.3% 

87.5% 

Lectures illustrated with multimedia eg; video clips 

34.3% 

59.4% 

Group discussions 

77.1% 

56.3% 

Questions and answers 

68.6% 

46.9% 

Tasks 

31.4% 

31.3% 

Other 

(please 

specify) 


14.3% 

5.7% 


Table 2. 

Percentage of respondents indicating that their recorded classes contain certain types of teaching formats, compared with the percentage of 
respondents indicating a preference for these formats to be included in recorded sessions. 


To improve the experience of the recorded classes, students identified better audio and image quality as 
being of primary importance (91 %). Some (30%) would like to see shorter edited recordings and some 
(27%) would prefer focused delivery of lecture content rather than class discussions, though comments 
were mixed, with some (distance learning) students noting strongly how important they find this 
opportunity. 
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The impact of watching the streamed classes on learning outcomes, assessed by retention of 
factual content 

Due to practical difficulties, some students were unable to engage with the classes in both study modes 
as scheduled. This meant that the group sizes were uneven, with 6 attending class 1, and 8 viewing that 
recording, 6 attending class 2 and 12 viewing the recording. Only 6 students completed both quizzes 
after attendance in each of the two modes, 3 in group A (attended first, viewed second) and 3 in group B 
(viewed first, attended second). 

As the sample size for the matched pairs analysis was reduced, we decided to undertake: 

(a) non-parametric statistical analysis in both with the matched sample (with n = 6); 

(b) and (b) an independent samples analysis using all complete quizzes. 

Using data from 7 students who took both tests after engaging with both classes in the same mode, (5 
'viewed' both classes and 2 'attended' both classes), there was no significant difference in median score 
between the two quizzes for these students (Wilcoxon test, a = 0.338, P = 0.735) suggesting that the two 
quizzes were comparable tests of information retention. 

Matched pairs analysis 

There was no significant difference (Wilcoxon z = 1.214, p = 0.225) between median scores following class 
attendance (14.25, 9.8-15.4 (IQR)) compared with those following viewing the streamed recording (13.0, 
9.6-14.0, n = 6). 

Independent samples analysis 

Again we could find no significant difference (Mann-Whitney u test u = 92.5, p = 0.289), between the 
median score for students who attended the class (12.1, 10.4-14.4, n = 12,) compared with that for 
students who viewed the recorded class (13.4, 12.9-14.0, n = 20). 

Conclusion 

Through a triangulation of three approaches, this study has demonstrated the potential positive impact 
that the recording and streaming of lectures on effective learning in a multi-campus university. The study 
has quantified the extent of current usage of the streaming technology at UHI and described both the 
positive aspects and the current constraints on the student experience of using streamed recordings. We 
have also demonstrated a robust methodology to compare learning outcomes in a class setting in order to 
evaluate such advances in technology. 

Implications for practice 

In summary, both students and staff were engaging with the recording technology during this pilot phase 
and most recordings were being used by some students. The pattern of use rose during the early part of 
the semester then tailed off before the end. This suggests that the peak in use before assessments, which 
has been reported elsewhere and which may have been expected from the student survey, did not take 
place. Barriers to this use include short storage times for recordings, or at least an expectation that storage 
times would be short. The issue of persistence of recordings throughout the semester should be addressed 
if this type of use is to be facilitated. 

Despite significant technical problems both with playing the recordings and with the audio and image 
quality of the product, there was a strong enthusiasm among students for the streamed video resource, 
both from those who had, and those who had not yet, used the resource. Significant upgrades to video 
conferencing hardware and to the means of storing recordings at the University of the Highlands and 
Islands during 2009-10 should have addressed some of these issues. 

If the technology is to be fully adopted by staff and students across the university, our student survey 
suggests there are significant training needs, which must now be addressed. Students must be made fully 
aware of the existence of the resource and the way to access recordings. Staff must also be made aware 
of the importance of distributing the link to the recorded classes in a timely manner. The ability to edit 
the recording and to provide index marks to identify significant points in the lecture would improve the 
experience of watching the recordings. 


14 



ENGSTRAND & HALL: THE USE OF STREAMED LECTURE RECORDINGS 


We suggest that practitioners should define the educational goals to be met by using the recordings, so 
they can identify the appropriate content and format for the recordings. At present, our recordings are 
being used mainly by students who attend VCs but who require a backup for classes that are missed or to 
recap on complex material. Although the class recordings may be enriched for this purpose by the further 
addition of multimedia resources, the product does not require major changes to suit this use. However, 
students who cannot attend classes at all (fully online students) are also users of the resource at the 
university, and the requirements of this group may be significantly different. 

We had anticipated that a shorter recording of a tutor delivering content directly to the camera may have 
been more appropriate for these students; however our study suggests that at least some of these students 
feel that they gain significantly from being able to see and hear class discussions. The lack of ability to 
interact is a significant concern and it is essential that these students are offered an alternative form 
of interaction with staff and peers in other components of their course. Discussion boards and quizzes 
are examples of asynchronous interactivity which are routinely used to engage online students. At UHI, 
synchronous virtual classes (facilitated using the Blackboard 'Chat' or 'Virtual Classroom' facility, or using 
technologies such as Skype, phone calls or audio conferences) are also currently used to give opportunities 
for questions, immediate feedback and peer interaction. 

Although our investigation into the retention of factual content was restricted by our small sample size, 
we felt that the methodology was sound and that the approach should be applied to a larger cohort to 
provide a more robust test. Our results were consistent with previous studies, which found that retention 
in students who have watched recordings of lectures was at least as high as those who attended the class 
(Akiyama, Teramoto and Kozono, 2008: McKinney, Dyck and Luber, 2009). This may suggest that concerns 
over the quality of the learning experience of students viewing recorded classes should focus on aspects of 
the experience other than retention, for example, engagement, enjoyment and group cohesion. 
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